B-spline Shape from Motion & Shading: An Automatic Free-form Surface Modeling for Face Reconstruction

Recently, many methods have been proposed for face reconstruction from multiple images, most of which involve fundamental principles of Shape from Shading and Structure from motion. However, a majority of the methods just generate discrete surface model of face. In this paper, Bspline Shape from Motion and Shading (BsSfMS) is proposed to reconstruct continuous B-spline surface for multiview face images, according to an assumption that shading and motion information in the images contain 1stand 0thorder derivative of B-spline face respectively. Face surface is expressed as a B-spline surface that can be reconstructed by optimizing B-spline control points. Therefore, normals and 3D feature points computed from shading and motion of images respectively are used as the 1stand 0thorder derivative information, to be jointly applied in optimizing the B-spline face. Additionally, an IMLS (iterative multileast-square) algorithm is proposed to handle the difficult control point optimization. Furthermore, synthetic samples and LFW dataset are introduced and conducted to verify the proposed approach, and the experimental results demonstrate the effectiveness with different poses, illuminations, expressions etc., even with wild images.